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ABSTRACT 



Problematizing complex datasets for students: Design principles for inquiry curriculum 

Josh Radinsky, Ben Loh, Jennifer Mundt, Sue Marshall, Louis M. Gomez, Brian J. 
Reiser, Daniel C. Edelson 

Northwestern University • Chicago Public Schools 

Radinsky, J., Loh, B., Mundt, J., Marshall, S., Gomez, L.M., Reiser, B.J., Edelson, D.C. 
(1999). Problematizing complex datasets for students: Design principles for inquiry 
curriculum. Paper presented at the Annual Conference of the American Educational 
Researchers Association, April 19-23, 1999, Montreal, Canada. 

ABSTRACT : 

Recent advances in computer technologies have provided new tools for accessing 
complex data for classroom inquiry. The effectiveness of these tools for learning 
depends upon the extent to which students learn to engage them reflectively. From an 
analysis of the design of an earth science inquiry unit using complex datasets, the authors 
(researchers and a middle school science teacher) propose four design principles that 
promise to be effective in promoting the development of reflective inquiry habits and 
skills. These are: (1) cultivating the need for data, (2) introducing data through students’ 
work, (3) promoting debate with evidence, and (4) building inquiry on earlier products of 
students’ work. 
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Introduction: the design challenge of complex datasets 

Recent advances in computer technologies, tele-communications, and learning sciences 
have resulted in a plethora of new tools for classroom inquiry. Tools such as data 
visualizers (e.g. Gordin & Pea, 1995), simulations (e.g. White & Frederiksen, 1995), and 
digital libraries and web browsers (e.g. Soloway, 1997) can bring unprecedented amounts 
of information directly to students and teachers in the classroom. While they represent 
exciting possibilities for teaching and learning through inquiry, there are many challenges 
involved in designing classroom activity around these sources of complex data. 

Classroom inquiry projects which use complex datasets often have one of two problems: 
either (1) students use only a small, pre-determined subset of the available data, and so do 
not develop skills and habits for making sense of complexity, or (2) students become lost 
in the information they see, failing to learn domain concepts or “discovering” false ones. 
As curriculum designers, we need to develop effective design approaches for helping 
students learn to make sense of complexity, balancing the need for structure with 
opportunities for freer exploration. 

In this paper we describe one product of our design research around this problem: the 
“Earth Structures and Processes” inquiry unit. This curriculum unit was developed by the 
authors and others as part of the Supportive Inquiry-Based Learning Environments 
project (Loh et al., 1997; Loh et al., 1998; Radinsky et al., 1998) at Northwestern 
University, in collaboration with the Center for Learning Technologies in Urban Schools. 
We have tried to create an inquiry curriculum that helps teachers give students extensive 
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guidance and practice in developing “ways of seeing” that are helpful for understanding 
complex data about earth structures and processes. At the same time, we have tried to 
design activities so that students take on increasing amounts of responsibility for making 
meaning of large sets of data themselves. We have tried to build in the need for students 
to make their own observations, questions, and explanations about data, supporting 
explanations with observable evidence. 

One key element in making sense of complexity is reflection. Our goal is to develop a set 
of curriculum design principles to help teachers and designers facilitate reflective inquiry 
with complex datasets. For the purposes of the design analysis presented here, we 
examine the kinds of reflection that were manifested in the enactment of the Earth 
Structures and Processes unit. We begin with a brief discussion to operationalize the 
construct of reflective inquiry. We then present an overview of the design of the 
curriculum unit. Next we discuss four design principles that emerged through the 
evolving design of the unit’s activities, over several iterations of design and classroom 
enactment. We use these design principles as a framework for presenting examples of 
student discourse from the most recent enactment of the unit, to illustrate relationships 
between designed elements of the curriculum, and particular kinds of reflective activity 
demonstrated by students 

What is reflective inquiry? 

What do we mean by reflective inquiry? Hawkins, Mawby and Ghitman (1987) identify 
reflection as the key element of critical inquiry: 

Critical inquiry skills are close kin to what has been described as 
reflective thinking — the ability to stand back from a topic or 
problem and reflect on it from a variety of perspectives... [This 
type of inquiry involves] active development of a question or 
problem, and exploration of information in order to find an answer 
or develop a connected, meaningful perspective... Previous 
assumptions are critically examined in the process, (p. 276) 

Two of the central concepts here — temporarily stepping back from ongoing work, and 
consideration from multiple perspectives — also are features of Collins and Brown’s 
(1988) discussion of ways that technology can promote learning through reflection on 
performances. 

Another feature of reflective inquiry suggested by this passage is that of active 
development of questions or problems. Lehrer and Romberg (1996), and Hiebert et al 
(1996) have characterized types of reflective questions generated by students and 
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teachers in conducting classroom inquiry, with a focus on reflection and the generation of 
valuable problems for investigation. In responses to this definition of reflection as 
probleniatizing. Smith (1997), Prawat (1997), and Hiebert et al (1997) suggest several 
aspects of inquiry activity that might be problematized in the classroom: for example, 
data points and representations, different possible problems within the situation, 
computational and problem-solving strategies, and students’ own conceptions related to 
the domain as they understand it. 

Also, the materials used in classroom activity can serve as another focus for reflection, as 
in the example of math manipulatives such as base 10 blocks (Fuson & Fraivillig, 1993). 
In this case, the materials are not to be considered problematic in their own right, but 
rather in their role as a model mediating between domain concepts (e.g. concepts of 
number) and items from experienced situations (e.g. cookies on a plate). 

Others have characterized reflective inquiry in more explicitly cognitive terms. For 
instance, Klahr Dunbar and Fay (1990) distinguish between data-driven inquirers who 
explore the “experiment space,” and more reflective inquirers who explore the 
“hypothesis space,” stepping back from work with data to consider multiple hypotheses 
that might be ruled out, or tested strategically. Schauble, Raghavan and Glaser (1993) 
state that “the specific importance of reflection is its role in consolidating the 
development of new strategies," linking reflection with the self- regulatory skills of 
maintaining goal-orientation (i.e. “holding in mind the goals and sub-goals [of] scientific 
discovery”), and self-evaluation during experimentation. 

1 he work presented here is part of a larger effort to develop a model of reflection in 
inquiry, in order to better understand the roles of reflection in teaching learning, and to be 
able to design inquiry curriculum which promotes reflection. Following the literature 
discussed above, we operationalize reflective inquiry as a mode of activity in which 
students: 

• periodically step back from ongoing activity with data, and make connections with a 
second perspective on their understanding of their current task, such as 

- domain concepts; 

- prior experiences, in or out of class; 

- goals of the current activity; or 

- expected outcomes of the current activity; and 

• generate questions, observations, predictions or explanations around artifacts of the 
inquiry process, such as: 
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- data points, such as dots on a map or numerical values in a list; 

- real-world referents of data, such as mountains or earthquakes; 

- models (physical or mental) representing relationships among data; 

- problems or problem representations; or 

- problem-solving strategies. 

This characterization of reflection in inquiry has guided our study of student activity over 
several cycles of design and enactment of the Earth Structures and Processes unit. Each 
iteration of the unit enabled us to explore the role of particular designs in promoting 
particular types of reflection. Therefore we structure our discussion of the kinds of 
reflection that happened around our current design rationale for the unit. 

Design of the curriculum 

This unit was designed for a b* or 7* grade class, to be conducting over 6 to 10 weeks. 
The unit covers content about the structure of the earth's crust, tectonic plate locations 
and movement, mechanisms of earthquakes and volcanoes, applying latitude and 
longitude, modeling earth structures and processes, map and globe reading, topography, 
and world geography. Data sources used in the unit include U.S. Geological Survey 
resources on the Internet, and the commercially available Geodynamics Database 
software. 

1. STAGING ACTIVITIES 

In skill-building and concept-building activities, students do mini-investigations of 
particular structures on the earth; create models of earth structures from topographical 
maps; and study and enact myths as explanations of earth structures. Map-making, 
plotting, and map usage teach concepts of graphic representations of data, for both 
continuous values (elevation) and point data (earthquake magnitude and depth). These 
staging activities introduce domain concepts both from the abstract and from the 
particular. 

Staging activities give students a chance to develop expertise through specific “case 
studies” of formations in the earth’s crust, examined through different points of view. 

For example: 

1.1 Earth structure expert groups 

Detailed information about 10 specific structures on the earth’s crust is brought into the 
classroom through student “expert groups.” Each group studies an information sheet 
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about their earth structure (see Figure 1), and teachers'can extend this research with 
suggested Internet or library resources if desired. 




Figure I. Earth structure information sheets. 



1.2 Modeling topographic data 

Observations and questions for inquiry are generated by creating and discussing 3D 
models of topographic maps of earth structures (see Figure 2). Making these models 
gives students a chance to interpret a dataset by creating another representation of it. The 
models also are meant to provide artifacts for later reflection, as students construct 
explanations of earth structures using their evolving domain understanding. 




Figure 2. Topographic maps and model of Japan 



2. INQUIRY CYCLES 

After a series of such staging activities, students begin cycles of inquiry in which they 
analyze earthquake and volcano data from a variety of sources. These investigations are 
interspersed with lectures and readings in preparation for the investigations; whole class 
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and small group discussion activities to explore concepts and understandings; and hands- 
on activities to build plotting and modeling skills. 

The first inquiry cycle involves gathering current earthquake data from the Internet and 
plotting it on a world map, to observe locations and measurements of tectonic activity 
and generate questions. In the second inquiry cycle students use a full year's earthquake 
data set to predict plate boundaries around the globe; students use a transparency overlay 
to draw their predictions, and use a “jigsaw” lesson structure and additional data to check 
their predictions. The third inquiry cycle uses the discovered plate boundaries, along with 
volcano and topographic data, to predict directions of plate motion; students create clay 
models representing plates, prominent earth structures, and plate motion. The complex 
data visualizations available in the Geodynamics Database software are used during the 
last two inquiry cycles. 

Inquiry cycles involve four kinds of activity: 

2.1 Creating datasets and looking for patterns 

Students create their own datasets initially by plotting the most recent earthquakes from 
the Internet (see Figure 3). This is extended by dividing up the world into regions; 
groups monitor and plot earthquake activity in their region for a period of time. In this 
way a large dataset is assembled by the class as a whole, which is then studied for 
patterns in the data. 




Figure 3. Earthquake data gathered and plotted by students. 

2.2 Making predictions from a large dataset 

After seeding discussions with the class-created datasets, a larger pre-designed dataset is 
introduced. Students draw predictions of plate boundaries on a plastic map overlay, 
using a data map showing one year’s dataset of large earthquakes worldwide (see Figure 

4). 
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Figure 4. Students marking boundaries that they are “Sure” of in one color, and “Not Sure" in another. 

2.3 Refining predictions with more data 

After using one large dataset for making concrete predictions, students are taught how to 
query much larger, interactive data sources. Students use Geodynamics Database 
software and other data sources to refine all groups’ predictions of plate boundaries, 
investigating the areas that were argued over in the previous phase of inquiry, and areas 
where predictions were less certain. 




Figure 5. A plastic map overlay showing students' revised predictions, alongside screens from the data 
visualization software. 

2.4 Representing findings ieads to new questions 

The refined predictions from phase 2.3 generate new, more sophisticated directions for 
students’ investigations. Models and images showing students’ plate boundary 
predictions and data interpretations are used to generate new questions about the 
directions of plate motion, requiring further data and another inquiry cycle. 
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Figure 6. Students ’ clay model of their plate, with a computer screen showing questions and observations 
about plate boundaries related to earthquake and volcano data. 



3. PRESENTATION OF FINDINGS 

Students create presentations and/or other instructional activities for a younger class, 
using the artifacts created in the unit to explain their understandings of earth structures 
and processes, and also summarizing their inquiry activities to show how they developed 
these understandings. 

There are two kinds of presentations that students do during the unit: 

3.1 Presentations during inquiry 

Groups compare their interpretations and findings with those of other groups (see Figure 
7), and identify areas that need more work. These presentations lead to further 
investigations with data, ad groups questions become more focused and refined around 
nP.P.HinQ to imHp.rsf^nH 
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Figure 7, Students present their plate boundary predictions to another group as part of a '‘mini- 
conference, ” 

3.2 Final presentations 

Groups use the artifacts that they have created (both computer-based and physical 
artifacts) to teach younger students about earth’s crustal structures and processes of 
change (see Figure 8). They explain their own learning process, as well as the domain 
concepts they have come to understand. 
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Figure 8. Students teach a group from a younger class about plate tectonics, using the artifacts of their 
inquiry (computer files and physical models). 



Design principles: Connecting designs to reflection 



The design principles proposed in this section have been emergent in our design work, 
and embody our working hypotheses about how curriculum designs can impact reflection 
in inquiry. We are currently developing a more complete framework for analyzing 
episodes of reflection in inquiry, to be presented elsewhere. The discussion here is to 
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designs which were intended to prompt such reflections. 



We use four emergent design principles to structure our presentation of the kinds of 
reflection that did occur in the course of enactment. The four principles are; 



• Cultivate the need for data 



• Introduce data through students’ work 

• Promote debate with evidence 



• Build inquiry on earlier artifacts 

Below we explore each of these principles as they played out in the design rationale and 
enactment of the unit. 



1 . Cultivating the need for data 

Students need to need the data, or they will not reflect on its meaning. By creating their 
own datasets to generate new questions for them, students approach other complex 
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datasets with a need to understand. This can lead them to develop their own high 
standards for quality work, as well as provide insights into what the complex data 
represent. 

To problematize the large sets of earthquake data, reflective discussions were used 
between the phases of the inquiry cycle described above. These discussions connected 
the questions and observations from one phase to the investigations of the next. For 
example, as students were introduced to the data available on the Internet, the teacher 
brought out a wide range of reflections about the data they saw, encouraging students to 
problematize the individual data points (all student names are pseudonyms): 

Jcanah: Can we have earthquakes in water? 

Anna: Ms. Murdt, vhy is there more earthquakes in the morning? 

These kinds of questions were revisited after the first datasets were created through 
student plotting of downloaded data. In the discussion between inquiry phases 2.1 and 
2.2, ideas were brought out that problematized patterns of data, seeking connections to 
possible explanations for the patterns: 

Anna: I t±iink naybe all the earthquakes are connected 

Arthur: [And there would be] aftershocks in Nigeria 

Renee: You could itake a graph of this! [earthquake data] 

Lanie: If we looked at 2 years ago, then last year's, then this 

year's ... 

Kenee: it we iookect in the ccnputer where the first earthquake 

was, when the date was... 

In these discussions, patterns are made problematic, setting the stage for looking for 
patterns in the larger dataset used in inquiry phase 2.3. Thus the problematizing done 
here sets the stage for later reflection back to these concepts from discussion. 

2. Introducing data through students’ work 

Rather than the teacher and the text being gatekeepers of all information, important data 
is brought into the classroom through the artifacts created by students. This can promote 
an environment in which students pay close attention to what one another are saying, and 
can relieve some of the burden on teachers to have to provide all of the relevant domain 
information. 

The unit provides lots of domain information within the materials used in the lessons 
themselves (in addition to pointers for further teacher learning in other places). For 
example, through the “Information Sheets” describing a range of real earth structures. 
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concrete domain facts are brought directly to the student “expert groups” for mini- 
research projects early in the unit. This means the teacher AND the other students can 
learn important information from their fellow classmates, as key concepts are introduced 
into classroom discussions through the students’ presentations. 

To make this kind of introduction of information effective, students must learn to reflect 
back on comments made by their classmates. This kind of referencing of each other did 
occur in the enactments. For example, certain pieces of information brought out in 
presentations of the earth structure expert groups were repeatedly mentioned later in the 
unit: 

Lanie: I ronember when Bobby said the Himalayas were graving 

sorething like 3 cm per year ... Ifeybe that's hew fast the 
plates are moving 

Mohantrad: If it grows 3 cm per year ... it will be bigger than the 
earth! 

Here we see one reflection which references an earlier experience (Bobby’s group’s 
presentation) to make sense of a current question (how fast plates move); and another 
reflection which problematizes the information presented by Bobby’s group. 

Also, students introduced another kind of information for each other: feedback on their 
evolving interpretations of data. This was accomplished through the mini-conferences 
that connected phases 2.2 - 2.4 of the inquiry cycle. This creates a more student- 
referenced environment than receiving feedback only from the teacher. It produced 

reflect! on R on each others’ intern ret atiens sneh ac the felle\x/ina* 

Gerald: V\te thought that [putting a boundary] here ... vjas the most 

accurate ... but then with Taniesha's group ... we found 
that there was a crack ... We proved that by looking at 
the cotputer data 

Bcfcby: For some earth structures ... you could have a subduction 

zone ... then ri^t next to it a transform zone 

Gerald: [We found] rtany buckling zones ... and no transform zones 

at all 

This design approach provides the teacher with tools for making students’ work relevant 
to the larger goals of the classroom inquiry. This usefulness of the information they 
produce seems helpful in promoting reflection, in the form of connecting current activity 
with prior classroom experiences, including recurring discussions of domain concepts 
and student predictions and interpretations. 
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3. Promoting debate with evidence 

Norms of classroom discussion and “table talk” during project work can be developed by 
the teacher, and supported by activities that distinguish between observations and 
explanations (see Kuhn, 1989, 1993, for theoretical discussions of these dispositions and 
skills). Assessment of observation habits rather than just correct answers, and activities 
that include repeated cycles of predictions and revisions, can help promote these norms. 

The unit uses “jigsaw” cooperative group structures to divide up places on earth which 
students will study, in several different phases of the unit. By having to match their 
predictions from one place with another group’s predictions from another, students are 
led to use the data as a basis for trying to improve their predictions so that all groups 
agree. Also, reflection questions for each activity point to the need to explain how 
assumptions are supported by evidence. 

Norms of teacher-facilitated discussion can promote argument from evidence rather than 
authority. These include encouraging students to explain and challenge ideas based on 
shared observations of data: 

Anna: [earthquakes] tas to do with the vvater ... 

Renee: California's dry, and it has earthquakes 

Ahad: That's because it's a tropical area 

MDhaitttHd: I thought earthquakes happoi ... whenever there's coasts 

Renee: Well Florida has an ocean by it, and there's no 

earthquakes 

Here Renee uses observable evidence to challenge her classmates’ explanations - a 
reflective act that connects concepts from discussion with explanatory domain models. 
By allowing these debates to proceed, instead of inserting the “correct” answers, the 
teacher uses the unit’s activities as a prompt to get students in the habit of connecting 
evidence to their explanations. When the teacher does intervene in a debate, it is to ask 
students to be more explicit in their observations or explanations; 



Arthur: 


Some of these earthquakes are related to each 
they're only 2 minutes apart 


other ... 


Lanie: 


There are patterns ... it happens in 
not wany h^pen alone ky themselves 


clusters or 


groups ... 


Teacher: 


do they cane in clusters? 






Lanie: 


... aftershocks 






'I^niesha: 


[if there are aftershocks after an 
does it step? 


earthquake] , 


so vshm 



The design of the unit’s activities requires teachers to push students to develop and 
defend their own interpretations. Open-ended interpretation tasks, like explaining the 
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formation of an earth structure, or predicting the boundaries of a plate, can promote the 
kinds of challenges raised by Renee and Taniesha above. However, we found that they 
are much more effective when the debates actually lead somewhere, such as debating 
plate boundaries before creating models of the plates based on those boundaries. The 
need for supporting evidence must be reinforced by the teacher throughout the unit. 
Otherwise, students will focus on who has the right answer, and will reason from 
authority rather than evidence. 

4. Building inquiry on earlier artifacts 

In order to reflect, students need artifacts to reflect back on — artifacts which capture their 
thinking and understandings. Inquiry cycles and presentations are sequenced so that 
students create artifacts in one activity which they will use as data in later activities. For 
example, after mapping separate plates, they create a world plate map as a whole class; 
this world plate map is the starting point for investigating directions of plate motion in 
the next inquiry cycle. Similarly, the models and data representations they create in each 
phase of the unit can be used to explain the domain concepts to an audience of younger 
students in the culminating activity. 

Reflecting back on their earlier artifacts was something that happened periodically during 
enactments, as when Bobby recalled his clay model during a discussion of plates: 

BnH-y Say for Japan, vdiere there's more depth than altitude, 
it might be that two plates collided, and they went dcwn 
- and that's how we got more d^th. 

We had hoped for more of these kinds of direct reflections connecting artifacts to 
understandings. Our current re-design of the unit has focused on developing the 
trajectory of student products to better support this kind of reflection. For example, 
groups will create all of their artifacts around the same earth structure, rather than 
skipping around to different areas. We hope that this will help them develop more of a 
coherent set of connections among the different types of data and models that they 
produce. 

In this sense, a clear trajectory of artifacts is closely related to what teachers described as 
a better “flow” to the unit - a sense of building from one activity to the next. If this flow 
of activities is supported by a coherent trajectory of increasingly-complex artifacts 
created by students, we hope to see increased reflections on these artifacts. We expect 
that this will also motivate students to focus more on the quality of their artifacts (if they 
must use them later), and will help build an environment in which revision is the norm. 
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Future directions 

These four design principles, and hypothesized connections to reflective student 
discourse, indicate the future directions of our program of design research. By making 
our assumptions explicit, and searching for mappings between design assumptions and 
reflection in activity, we hope to begin a detailed investigation of the role of particular 
design elements in fostering reflective classroom inquiry. 

A closer analysis of the trajectories of change in reflective discourse for particular 
students will give us more feedback on the actual impact of the designs. In analysis of 
reflective episodes around each of the major activities of the unit, we will look for 
evidence of the ways students refer to particular artifacts in reflections connecting one 
activity context with another. We expect that different students have different 
dispositions for reflection at the outset, and thus will change along different trajectories. 

The concept of students’ dispositions for reflective inquiry is closely tied to Rogoff’s 
(1995) concept of roles for participation in group activity. One of our larger goals with 
this analysis will be to begin mapping out different trajectories of change as students 
adopt new roles in their group work, with respect to activity around complex data. By 
starting from our design rationale, and mapping our assumptions to actual roles adopted 
by students, we look for connections between particular design approaches, and particular 
changes in the disposition to reflect. We feel that this approach has great promise for 
making sense of the complex terrain that is the design of curriculum. 
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Earth Structures and Processes: 

Exploring the earth’s crust with models and data 

http://www.letus.nwu.edu/projects/esp/esp.html 
A thematic science unit from the Center for Learning Technologies in Urban Schools 



Northwestern University: Josh Radinsky, Ben Loh, Sue Marshall - SIBLE Project 
Chicago Public Schools: Jennifer Mundt - Boone Elementary, SIBLE Project 

Judith Lachance-Whitcomb - Sauganash Elementary 
Lou Ellen Finn, Thea Raedeke - Hayt Elementary 
Jean Bramlette - Haines Elementary 

Sonia Flore, Kathleen North-Tomczyk - Saucedo Academy 

Jennifer Olson - Inter-American Magnet School 

The central “seed” activity for this unit was designed by Ben Loh for the SIBLE project 
< http://www.ls.sesp.nwu.edu/sible/ >. It was later adapted and conducted with a 7‘^ grade 
class over 2 weeks by Jennifer Mundt (Boone Elementary, CPS), Ben Loh, and Josh 
Radinsky. Some initial materials and lesson plans were generated in the course of this 
enactment. 

The roiiowing semester this seed activity was Drougnt to a work circle or teacners from 
two schools, including Jean Bramlette (Haines Elementary, CPS), Sonia Flores (Saucedo 
Academy, CPS), Kathleen North-Tomczyk (Saucedo Academy, CPS), working with Josh 
Radinsky. A variety of staging activities were drafted by this group, and conducted with 
students from an after-school club in one school, and a 7*'' grade class in the other. Work 
circle members alternated teaching the activities and observing one another, later 
reflecting on how to revise and write up each activity. Drafts of a unit outline and 
specific activities were presented to meetings of science teachers from 6‘^, 7‘^ and 8"' 
grades at eachwork circle school for feedback. 

Building on these mini-enactments and revisions, the complete unit materials were then 
written and revised by Jennifer Mundt, Josh Radinsky, Judith Lachance-Whitcomb 
(Sauganash Elementary, CPS) and LouEllen Finn (Hayt Elementary, CPS) over the 
summer. This completed draft of a 6 — 8 week unit was piloted in about 10 classrooms 
during the school year, and studied in depth in three classrooms by Josh Radinsky 
working with Jennifer Mundt, Jennifer Olson (Inter-American Magnet, CPS), and Thea 
Raedeke (Hayt Elementary, CPS). 
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These pilot studies led to extensive revisions of the unit, and culminated in a unit-revision 
meeting to compile ideas for improving the content, structure, and flow of the unit. The 
revised unit will be available for the 1999-2000 academic year. 




19 



16 



Radinsky et al 



Probleniatizing complex datasets for students 



AERA 1999 



References 

Collins, A., & Brown, J. S. (1988). The computer as a tool for learning through 
reflection. In H. Mandl & A. Lesgold (Eds.), Learning issues for intelligent 
tutoring systems , (pp. 1-18). New York: Springer-Verlag. 

Fuson, K. C., & Fraivillig, J. L. (1993). Supporting children’s ten-structured thinking 
in the classroom. In G. Bell (Ed.), Asian Perspectives on Mathematics Education . 

. Melbourne: Australian Council for Educational Research. 

Gordin, D. N., & Pea, R. D. (1995). Prospects for scientific visualization as an 
educational technology. The Journal of the Learning Sciences. 4 . 249-279. 

Hawkins, J., Mawby, R., & Ghitman, J. M. (1987). Practices of novices and experts in 
critical inquiry. In R. D. Pea & K. Sheingold (Eds.), Mirrors of minds: Patterns of 
experience in educational computing . . Norwood, NJ: Ablex. 

Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., Olivier, 
A., & Weame, D. (1996). Problem solving as a basis for reform in curriculum and 
instruction: the case of mathematics. Educational Researcher. 25 (4). 12-21. 

Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., Olivier, 

A r> "nx <.i . * ; _ ; 

i'v., OC tVv^cuiiv^, 1--^. ! y. ivxarwiiig pi rs. iLjuiiiUL/i lu 

Prawat and Smith. Educational Researcher. 26 (2). 24-26. 

Klahr, D., Dunbar, K., & Fay, A. L. (1990). Designing good experiments to test bad 
hypotheses. In J. Shrager & P. Langley (Eds.), Computational models of scientific 
discovery and theory formation , (pp. 355-402). Palo Alto, CA: Morgan Kaufmann 
Publishers, Inc. 

Kuhn, D. (1989). Children and adults as intuitive scientists. Psychological Review. 96 . 
674-689. 

Kuhn, D. (1993). Connecting scientific and informal reasoning. Merrill-Palmer 
Quarterly, 39 (1). 74-103. 

Lehrer, R., & Romberg, T. (1996). Exploring children's data modeling. Cognition and 
Instruction. 14 (1). 69-108. 



Loh, B., Radinsky, J., Reiser, B. J., Gomez, L. M., Edelson, D. C., & Russell, E. 
(1997, ). The Progress Portfolio: Promoting reflective inquiry in complex 



Radinsky et at 



A ERA 1999 



Problematizing complex datasets for students 



investigation environments. Paper presented at the Proceedings of Computer 
Supported Collaborative Learning '97, Toronto, Ontario, Canada. 

Loh, B., Radinsky, J., Russell, E., Gomez, L. M., Reiser, B. J., & Edelson, D. C. 
(1998, April, 1998). The Progress Portfolio: Designing reflective tools for a 
classroom context. Paper presented at the Proceedings of CHI 98, Los Angeles, 
CA. 

Prawat, R. S. (1997). Problematizing Dewey's views of problem solving: A reply to 
Hiebert et al. Educational Researcher. 26 (2'). 19-21. 

Radinsky, J., Loh, B., Brown, M., Reiser, B., Gomez, L., & Edelson, D. (1998, April, 
1998). Strategic design dimensions for promoting reflection in complex data sets. 
Paper presented at the Annual Conference of the American Educational 
Researchers Association, San Diego, CA. 

Rogoff, B. (1995). Observing sociocultural activity on three planes: participatory 
appropriation, guided participation, and apprenticeship. In J. V. Wertsch, P. d. 
Rfo, & A. Alvarez (Eds.), Sociocultural studies of mind . . New York: Press 
Syndicate, University of Cambridge. 

Schauble, L., Raghavan, K., & Glaser, R. (1993). The discovery and reflection 

iiuiaiiuii; A giapliii;ai iiai;c lui suppuiimg scif-icguiaiiuii iii (juiiiputci -ua&cu 

laboratories. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools , 
(pp. 319-337): Erlbaum. 

Smith, J. P. (1997). Problems with problematizing mathematics: A reply to Hiebert et 
al. Educational Researcher. 26 (2). 22-24. 

Soloway, E. a. W., Raven,. (1997). Does the internet support student inquiry? Don't 
ask. Communications of the ACM. 40 (5). 11-16. 

White, B. Y., & Frederiksen, J. R. (1995). An overview of the ThinkerTools inquiry 
project (Causal Models Report 95-04): University of California, Berkeley. 



ERIC 



Zi 



18 



f 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 







TM031807 



I. DOCUMENT IDENTIFICATION: 



Title: ^ ) jl,' t\ 


pn'hW|>4;j 




Authorise 




Corporate Source: 




Publication Date: 



II. REPRODUCTION RELEASE: 

In onler to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system. Resources in Education (RIE), are usually made available to users in miaofiche. reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 



of the page. 

The sample sticker shown below will be 
affixed to all Level 1 documents 


The sample sticker shown below will be 
affixed to an Level 2A documents 


The sample sticker shown below will be 
affixed to aSi Level 2B documents 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA* 
FOR ERIC COLLECTION SUBSCRIBERS ONLY. 
HAS BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 


















c# 


'-o'' 

TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


1 




2A 




2B 



Level 1 



Level 2A 



Level 2B 






□ 




Check here for Level 1 lelease, pemtitting 
reproduction and dissemination In microfiche or other 
ERIC archival media (e.g., electronic) and paper 
copy. 



Check hem for Level 2A release, permitting Check here for level 2B release, permitting 

reproduction and dissemination In microfiche and In reproduction and rfissemination In microfiche only 

electronic media for ERIC archival collection 
subscribers only 



Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box Is checked, documents wid be processed at Level 1. 



Sign 

here,-^ 

please 



U 




/ hereby grant to the Education^ Resources information Center (ERiC) nonexclusive pennission to reproduce and disseminate this document 
as Indicated above. Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from the copyright holder. Exception Is made for non-profit reproduction by Otraries and other service agencies 
to satisfy Information needs of educators In response to discrete inquiries. 


L <0-^ 


Prkited Name/Positionfritle: _ 


Organtzation/Address:^ ^ 

0»uiv</5rf^ <»r Jllir\0ii 

{okO cJ. i-L{f(ii.tyA, COJ{C.Ajy> iL 


Telephone: . ^ ^ . 


‘f‘)6'6foO 


E-M^l Address: _ . . ^ /) 





(over) 




